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Abstract 

A search for the pair production of fourth generation fe'-quarks was performed 
using data taken by the DELPHI detector at LEP-II. The analysed data were 
collected at centre-of-mass energies ranging from 196 to 209 GeV, corresponding 
to an integrated luminosity of 420 pb _1 . No evidence for a signal was found. 
Upper limits on BR(b' — ► bZ) and BR{b' — > cW) were obtained for b' masses 
ranging from 96 to 103 GeV/c 2 . These limits, together with the theoretical 
branching ratios predicted by a sequential four generations model, were used to 
constrain the value of Rckm = \ v ch v 1> wnere Kb', Kb' and V t b are elements 
of the extended CKM matrix. 
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1 Introduction 

The Standard Model (SM), although in agreement with the available experimental 
data [1], leaves several open questions. In particular, the number of fermion generations 
and their mass spectrum are not predicted. The measurement of the Z decay widths [1] 
established that the number of light neutrino species (m < mz/2, where mi is the Z boson 
mass) is equal to three. However, if a heavy neutrino or a neutrinoless extra generation 
exists, this bound does not exclude the possibility of extra generations of heavy quarks. 
Moreover the fit to the electroweak data [2] does not deteriorate with the inclusion of one 
extra heavy generation, if the new up and down-type quarks mass difference is not too 
large. It should be noticed however that in this fit no mixing of the extra families with 
the SM ones is assumed. 

The subject of this paper is the search for the pair production of a fourth generation 
b'-quark at LEP-II: b' production and decay are discussed in section 2; in section 3, the 
data sets and the Monte Carlo (MC) simulation are described; the analysis is discussed 
in section 4; the results and their interpretation within a sequential model are presented 
in sections 5 and 6, respectively. 

2 b'-quark production and decay 

Extra generations of fermions are predicted in several SM extensions [3,4]. In sequen- 
tial models [5-7], a fourth generation of fermions carrying the same quantum numbers as 
the SM families is considered. In the quark sector, an up-type quark, t', and a down-type 
quark, b', are included. The corresponding 4x4 extended Cabibbo-Kobayashi-Maskawa 
(CKM) matrix is unitary, approximately symmetric and almost diagonal. As CP-violation 
is not considered in the model, all the CKM elements are assumed to be real. 

The b'-quark may decay via charged currents (CC) to UW, with U = t', t, c, u, or via 
flavour- changing neutral currents (FCNC) to DX, where D = b, s, d and X = Z,H,7,g 
(Fig. 1). As in the SM, FCNC are absent at tree level, but can appear at one-loop level, 
due to CKM mixing. If the b' is lighter than t' and t, the decays b' — > t'W and b' — > tW 
are kinematically forbidden and the one-loop FCNC decays can be as important as the 
CC decays [6]. 

The analysis of the electroweak data [1] shows that the mass difference \m t i — my\ < 
60 GeV/c 2 is consistent with the measurement of the p parameter [3,5]. In particular, 
when m z + m b < my < m H + m h , either b' — > cW or b' — * bZ decay tend to be domi- 
nant [5-7]. In this case, the partial widths of the CC and FCNC b' decays depend mainly 
on m t >, my and Rqkm = \ \^% b I ? where V c y, V t y and 14b are elements of the extended 
4x4 CKM matrix [7]. 

Limits on the mass of the b'-quark have been set previously at various accelerators. 
At LEP-I, all the experiments searched for b' pair production (e + e~ — > b'b'), yielding a 
lower limit on the b' mass of about m z /2 [8]. At the Tevatron, both the DO [9] and 
CDF [10] experiments reported limits on <r(pp — > b'b') x BR(h' — > bX) 2 , where BR is 
the branching ratio corresponding to the considered FCNC b' decay mode and X = 7, Z. 
Assuming BR{b' — > bZ) = 1, CDF excluded the region 100 < my < 199 GeV/c 2 . 
Although no dedicated analysis was performed for the b' — > cW decay, the DO limits on 
<r(pp — > tt) x BR(t — > cW) 2 from Fig. 44 and Table XXXI of reference [11] can give a 
hint on the possible values for BR(W — > cW) [12]. 
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In the present analysis the on-shell FCNC (b' — > bZ) and CC (b' — > cW) decay modes 
were studied and consequently the mass range 96 GeV/c 2 < my < 103 GeV/c 2 was 
considered. This mass range is complementary to the one covered by CDF [10]. The 
mass range m\y + m c < my < mz + m\, was not considered because in this region 
the evaluation of the branching ratios for the different b' decays is particularly difficult 
from the theoretical point of view [7]. In the present analysis no assumptions on the 
BR(h' — > bZ) and BR(h' — > cW) in order to derive mass limits were made. Different 
final states, corresponding to the different b' decay modes and subsequent decays of the 
Z and W bosons, were analysed. 

3 Data samples and Monte Carlo simulation 

The analysed data were collected with the DELPHI detector [13] during the years 
1999 and 2000 in LEP-II runs at y/s = 196 — 209 GeV and correspond to an integrated 
luminosity of about 420 pb _1 . The luminosity collected at each centre-of-mass energy is 
shown in Table 1. During the year 2000, an unrecoverable failure affected one sector of 
the central tracking detector (TPC), corresponding to 1/12 of its acceptance. The data 
collected during the year 2000 with the TPC fully operational were split into two energy 
bins, below and above y/s = 206 GeV, with (y/s) = 204.8 GeV and (y/s) = 206.6 GeV, 
respectively. The data collected with one sector of the TPC turned off were analysed 
separately and have (y/s) = 206.3 GeV. 



(GeV) 


196 


200 


202 


205 


207 


206* 


luminosity (pb _1 ) 


76.0 


82.7 


40.2 


80.0 


81.9 


59.2 



Table 1: The luminosity collected with the DELPHI detector at each centre-of-mass 
energy is shown. The energy bin labelled 206* corresponds to the data collected with one 
sector of the TPC turned off. 

Signal samples were generated using a modified version of PYTHIA 6.200 [14]. Al- 
though PYTHIA does not provide FCNC decay channels for quarks, it was possible to 
activate them by modifying the decay products of an available channel. The angular 
distributions assumed for b' pair production and decay were those predicted by the SM 
for any heavy down-type quark. Different samples, corresponding to b' masses in the 
range between 96 and 103 GeV/c 2 and with a spacing of 1 GeV/c 2 were generated at 
each centre-of-mass energy. Specific Monte Carlo simulations (for both SM and signal 
processes) were produced for the period when one sector of the TPC was turned off. 

The most relevant background processes for the present analyses are those leading 
to WW or ZZ bosons in the final state, i.e. four-fermion backgrounds. Radiation in 
these events can mimic the six-fermion final states for the signal. Additionally (7(7(7) and 
Bhabha events can not be neglected since for signal final states with missing energy these 
backgrounds can become important. SM background processes were simulated at each 
centre-of-mass energy using several Monte Carlo generators. All the four-fermion final 
states (both neutral and charged currents) were generated with WPHACT [15], while 
the particular phase space regions of e + e~ — > e + e~ff referred to as 77 interactions were 
generated using PYTHIA [14]. The qq(7) final state was generated with KK2F [16]. 
Bhabha events were generated with BHWIDE [17]. 
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The generated signal and background events were passed through the detailed simu- 
lation of the DELPHI detector [13] and then processed with the same reconstruction and 
analysis programs as the data. 

4 Description of the analyses 

Pair production of b'-quarks was searched for in both the FCNC (b' — > bZ) and CC 
(b' — > cW) decay modes. The b' decay modes and the subsequent decays of the gauge 
bosons (Z or W) lead to several different final states (Fig. 2). The final states considered 
and their branching ratios are shown in Table 2. The choice of the considered final 
states was done taking into account their signatures and BR. About 81% and 90% of the 
branching ratio to the FCNC and CC channels were covered, respectively. All final states 
include two jets originating from the low energy b (c) quarks present in the FCNC (CC) 
b' decay modes. A common preselection was adopted, followed by a specific analysis for 
each of the final states (Table 2). 



b' decay 


boson decays 


BR (%) 


final states 


b' -> bZ (FCNC) 


ZZ -> \n~up 


4.0 


bbl+l-i/P 




ZZ — > qquu 


28.0 


bbqqz/z/ 




ZZ -> qqqq 


48.6 


bbqqqq 


b' -> cW (CC) 


WW -> qql+z/ 


43.7 


ccqql + z/ 




WW — > qqqq 


45.8 


ccqqqq 



Table 2: The final states considered in this analysis are shown. About 81% and 90% of 
the branching ratio to the FCNC and CC channels were covered, respectively. 

Events were preselected by requiring at least eight good charged-particle tracks and 
the visible energy measured at polar angles 1 above 20°, to be greater than 0.2y/s. Good 
charged-particle tracks were defined as those with a momentum above 0.2 GeV/c and 
impact parameters in the transverse plane and along the beam direction below 4 cm and 
below 4 cm/ sin#, respectively. 

The identification of muons relied on the association of charged particles to signals 
in the muon chambers and in the hadronic calorimeters and was provided by standard 
DELPHI algorithms [13]. The identification of electrons and photons was performed 
by combining information from the electromagnetic calorimeters and the tracking sys- 
tem. Radiation and interaction effects were taken into account by an angular clustering 
procedure around the main shower [18]. 

The search for isolated particles (charged leptons and photons) was done by construct- 
ing double cones oriented in the direction of charged-particle tracks or neutral energy 
deposits. The latter ones were defined as calorimetric energy deposits above 0.5 GeV, 
not matched to charged-particle tracks and identified as photon candidates by the stan- 
dard DELPHI algorithms [13,18]. For charged leptons (photons), the energy in the region 
between the two cones, which had half-opening angles of 5° and 25° (5° and 15°), was 
required to be below 3 GeV (1 GeV), to ensure isolation. All the charged-particle tracks 

1 In the standard DELPHI coordinate system, the positive z axis is along the electron beam direction. The polar angle 
(0) is defined with respect to the z axis. In this paper, polar angle ranges are always assumed to be symmetric with respect 
to the = 90° plane. 
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nnal state 


assignment criteria 


bbl + l - ^i? 


at least 1 isolated lepton 


bbqqz/z/ 


no isolated leptons 

E m issing > 50 GeV 


bbqqqq 


no isolated leptons 

Emissing ^ 50 GeV 


ccqql + z/ 


only 1 isolated lepton 


ccqqqq 


no isolated leptons 

Emissing < 50 GeV 



Table 3: Summary of the final state assignment criteria. 

and neutral energy deposits inside the inner cone were associated to the isolated particle. 
Its energy was then re-evaluated as the sum of the energies inside the inner cone and 
was required to be above 5 GeV. For well identified leptons or photons [13,18] the above 
requirements were weakened. In this case only the external cone was used (to ensure 
isolation) and its angle a was varied according to the energy of the lepton (photon) can- 
didate, down to 2° for P e > 70 GeV/c (3° for P 7 > 90 GeV/c), with the allowed energy 
inside the cone reduced by sina/ sin 25° (since/ sin 15°). Isolated leptons were required 
to have a momentum greater than 10 GeV/c and a polar angle above 25°. Events with 
isolated photons were rejected. 

All the events were clustered into two, four or six jets using the Durham jet algo- 
rithm [19], according to the number of jets expected in the signal in each of the final 
states, unless explicitly stated otherwise. Although two b jets are always present in the 
FCNC final states, they have a relatively low energy and b-tagging techniques [20] were 
not used. 

Events were assigned to the different final states according to the number of isolated 
leptons and to the missing energy in the event, as detailed in Table 3. Within the same 
b' decay channel, the different selections were designed to be mutually exclusive. For the 
final states involving charged leptons (bbl + Ez/z/ and ccqql + z/), events were divided into 
different samples according to the lepton flavour identification: e sample (well identified 
electrons), fi sample (well identified muons) and no-id sample (leptons with unidentified 
flavour or two leptons identified with different flavours). 

Specific analyses were then performed for each of the final states. The selection criteria 
for the bbqqqq and ccqqqq final states were the same. The hh\ + \~ uu final state has a very 
clean signature (two leptons with m\+\- ~ m^, two low energy jets and missing mass close 
to mz) and consequently a sequential cut analysis was adopted. For all the other final 
states, a sequential selection step was followed by a discriminant analysis. In this case, 
a signal likelihood (Cs) and a background likelihood (Cb) were assigned to each event, 
based on Probability Density Functions (PDF), built from the distributions of relevant 
physical variables. The discriminant variable was defined as ln(£s / ' Cb)- 

4.1 The hh\ + \~ui> final state 

The FCNC hYA + \~vu final state events were preselected as described above, by re- 
quiring at least eight good charged-particle tracks, the visible energy measured at polar 
angles above 20°, to be greater than 0.2^ and at least one isolated lepton. Distribu- 
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tions of the relevant variables are shown in Fig. 3 for all the events assigned to this final 
state after the preselection. The event selection was performed in two levels. In the first 
one, events were required to have at least two leptons and an effective centre-of-mass 
energy [21], V^, below 0.95a/s. The particles other than the two leptons in the events 
were clustered into two jets and the Durham resolution variable in the transition from 
two jets to one jet 2 was required to be greater than 0.002. The number of data events and 
the SM expectation after the first selection level is shown in Table 4. The background 
composition and the signal efficiencies at this level of selection for my = 100 GeV/c 2 and 
y/s = 205 GeV are given in Table 8. The efficiencies for the other relevant b' masses and 
y/s values were found to be the same within errors. Data, SM expectation and signal 
distributions at this selection level are shown in Fig. 4. 



(GeV) 


data (SM expectation ± statistical error) 




e sample 


/j, sample 


no-id sample 


196 


2 (2.6±0.3) 


1 (2.9±0.3) 


47 (35.9±1.4) 


200 


3 (2.5±0.4) 


4 (3.4±0.4) 


30 (37.4±1.4) 


202 


2 (1.3±0.2) 


1 (1.7±0.2) 


20 (18.7±0.7) 


205 


5 (2.5±0.4) 


3 (3.0±0.4) 


35 (36.2±1.4) 


207 


3 (2.3±0.4) 


3 (3.1±0.4) 


45 (35.1±1.3) 


206* 


1 (1.9±0.3) 


2 (2.6±0.2) 


31 (27.6±1.0) 


total 


16 (13.2±0.8) 


14 (16.7±0.8) 


208 (191.0±3.0) 



Table 4: First selection level of the bbl + l" vv final state: the number of events selected 
in data and the SM expectations after the first selection level for each sample and cen- 
tre-of-mass energy are shown. 

In the final selection level the momentum of the more energetic (less energetic) jet 
was required to be below 30 GeV/c (12.5 GeV/c). Events in the e and no-id samples 
had to have a missing energy greater than 0Ay/s. In the /x sample events were required 
to have an angle between the two muons greater than 125°. In the no-id sample, the 
angle between the two charged leptons had to be greater than 140° and p m i S /E mis < 0.4, 
where p m i S and E rnis are the missing momentum and energy, respectively. After the final 
selection, one data event was selected for an expected background of 1.5±0.7. This event 
belonged to the no-id sample and was collected at y/s = 200 GeV. The signal efficiencies 
for my = 100 GeV/c 2 and = 205 GeV are 30.6 ± 2.5% (e sample), 48.6 ± 2.7% (/i 
sample) and 7.2 ± 0.8% (no-id sample) and their variation with my and y/s was found 
to be negligible in the relevant range. 

4.2 The bbqqVz/ final state 

The FCNC bbqqz/z/ final state is characterised by the presence of four jets and a 
missing mass close to mz- At least 20 good charged-particle tracks and y/s' > 0.5^ 
were required. Events were clustered into four jets. Monojet-like events were rejected by 
requiring — log 10 (y2->-i) < 0.7 (?/2->i is the Durham resolution variable in the two to one 
jet transition). Furthermore, — log 10 (y4_ > 3) was required to be below 2.8 and the energy 
of the leading charged particle of the most energetic jet was required to be below 0.1 y/s. 

2 The Durham resolution variable is the minimum value of the scaled transverse momentum obtained in the transition 
from n to n — 1 jets [19] and will be represented by j/ n _>„_i. 
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A kinematic fit imposing energy-momentum conservation and no missing energy was 
applied and the background-like events with x 2 /n.d.j '. < 6 were rejected. The data, 
SM expectation and signal distributions of this variable are shown in Fig. 5. Table 5 
summarizes the number of selected data events and the SM expectation. The background 
composition and the signal efficiency at this level of selection for my = 100 GeV/c 2 and 
y/s = 205 GeV are given in Table 8. The efficiencies for the other relevant b' masses and 
y/s values were found to be the same within errors. 



(GeV) 


data (SM expectation ± statistical error) 


196 


123 (106.3±4.0) 


200 


111 (104.8±4.0) 


202 


50 (49.8±1.9) 


205 


88 (94.2±3.7) 


207 


99 (91.2±3.6) 


206* 


62 (65.7±2.6) 


total 


533 (511.7±8.3) 



Table 5: First selection level of the bbqqz/z/ final state: the number of events selected in 
data and the SM expectation for each centre-of-mass energy are shown. 

A discriminant selection was then performed using the following variables to build the 
PDFs: 

• the missing mass; 

• A 3 £g x min(sin^j 1 ,sin^j 2 ), where A™g is the acoplanarity 3 and 0j lt j 2 are the polar 
angles of the jets when forcing the events into two jets 4 ; 

• the acollinearity between the two most energetic jets 5 with the event particles clus- 
tered into four jets; 

• the sum of the first and third Fox- Wolfram moments (hi + h 3 ) [22]; 

• the polar angle of the missing momentum. 

The data, SM expectation and signal distributions of these variables are shown in Fig. 6. 

4.3 The bbqqqq final state 

The FCNC bbqqqq final state is characterised by the presence of six jets and a small 
missing energy. All the events were clustered into six jets and only those with at least 
30 good charged-particle tracks were accepted. Moreover, events were required to have 
> 0.Q\/s, — log 10 (y2->i) < 0.7 and — log 10 (t/6->5) < 3.6. The number of selected data 
events and the expected background at this level are shown in Table 6. The background 
composition and the signal efficiency at this level of selection for my = 100 GeV/c 2 and 
\/s = 205 GeV are given in Table 8. The efficiencies for the other relevant b' masses and 
t/s values were found to be the same within errors. 

A discriminant selection was performed using the following variables to build the PDFs: 

3 The acoplanarity between two particles is denned as |180° — \<f>\ — <f>2\\, where </>i,2 are the azimuthal angles of the two 
particles (in degrees). 

4 While the signal is characterised by the presence of four jets in the final state, the two jets configuration is used mainly 
for background rejection. 

5 The acollinearity between two particles is defined as 180° — 01,2, where 01,2 is the angle (in degrees) between those 
two particles. 
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Js (GeV) 

v v / 


data (SM expectation ± statistical error) 


196 


349 (326.7±5.3) 


200 


347 (342.1±5.5) 


202 


165 (162.1±2.6) 


205 


322 (319.0±5.2) 


207 


287 (307.6±5.0) 


206* 


192 (215.8±3.6) 


total 


1662 (1673.9±11.4) 



Table 6: First selection level of the bbqqqq and ccqqqq final states: the number of events 
selected in data and the SM expectations for each centre-of-mass energy are shown. 

• the Durham resolution variable, — log 10 (7/4-^.3); 

• the Durham resolution variable, — log 10 (7/5^.4); 

• the acollinearity between the two most energetic jets, with the event forced into four 
jets; 

• the sum of the first and third Fox- Wolfram moments; 

• the momentum of the most energetic jet; 

• the angle between the two most energetic jets (with the events clustered into six 
jets). 

The distributions of these variables are shown in Fig. 7 for data, SM expectation and 
signal. 

4.4 The ccqql + z/ final state 

The signature of this CC final state is the presence of four jets (two of them having 
low energy), one isolated lepton and missing energy (originating from the W — > IP decay). 
The events were accepted if they had at least 15 good charged-particle tracks. The event 
particles other than the identified lepton were clustered into four jets. Part of the qq 
and 77 background was rejected by requiring — log 10 (y 2 -»i) < 0.7. Furthermore, there 
should be only one charged-particle track associated to the isolated lepton, and the leading 
charged particle of the most energetic jet was required to have a momentum below 0.1 ^fs. 
The number of selected data events and SM expectations at this level are summarized in 
Table 7. The background composition and the signal efficiencies at this level of selection 
for my = 100 GeV/c 2 and y/s = 205 GeV are given in Table 8. The efficiencies for the 
other relevant b' masses and ^/s values were found to be the same within errors. 

The PDFs used to calculate the background and signal likelihoods were based on the 
following variables: 

• the sum of the first and third Fox- Wolfram moments; 

• the invariant mass of the two jets, with the event particles other than the identified 
lepton clustered into two jets; 

• the Durham resolution variable, — log 10 (2/4^3); 

• £\ \pi\/y/s, where Pi are the momenta of the charged particles (excluding the lepton) 
in the same hemisphere as the lepton (the hemisphere is defined with respect to the 
lepton); 

• the acollinearity between the two most energetic jets; 
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y/s (GeV) 

V V / 


data (SM expectation ± statistical error) 




e 


H 


no-id 


196 


65 (51.1±1.4) 


53 (56.1±1.5) 


38 (34.4±1.4) 


200 


54 (58.1±1.7) 


63 (59.9±1.6) 


40 (35.0±1.4) 


202 


30 (27.8±0.8) 


21 (28.4±0.8) 


13 (16.9±0.7) 


205 


56 (50.8±1.5) 


66 (53.6±1.5) 


32 (33.3±1.4) 


207 


53 (53.8±1.6) 


48 (57.2±1.6) 


35 (33.8±1.4) 


206* 


31 (37.2±1.4) 


42 (39.3±1.1) 


21 (23.4±1.0) 


total 


289 (278.8±3.5) 


293 (294.5±3.4) 


179 (176.8 ± 2.8) 



Table 7: First selection level of the ccqql + z/ final state: the number of events selected in 
data and the SM expectations for each sample and centre-of-mass energy are shown. 

• the angle between the lepton and the missing momentum. 

The data, SM expectation and signal distributions of these variables are shown in Fig. 8. 

In order to improve the efficiency, events with no leptons seen in the detector were 
kept in a fourth sample. For this sample, the selection criteria of the bbqqz/z/ final state 
were applied and the same variables as in section 4.2 were used to build the PDFs. The 
signal efficiency after the first selection level for my = 100 GeV/c 2 and y/s = 205 GeV 
was 8.9 ±0.9%. The efficiencies for the other relevant b' masses and y/s values were found 
to be the same within errors. 

4.5 The ccqqqq final state 

This final state is very similar to bbqqqq (with slightly different kinematics due to the 
mass difference between the Z and the W). The analysis described in section 4.3 was 
thus adopted. The number of selected events and the SM expectations can be found in 
Table 6. At this level, the signal efficiency for my = 100 GeV/c 2 and y/s = 205 GeV was 
67.3 ± 1.5%. The efficiencies for the other b' masses and centre-of-mass energies were the 
same within errors. The PDFs were built using the same set of variables as in section 4.3. 

5 Results 

For all final states, a good agreement between data and SM expectation was found. The 
summary of the total number of selected data events, SM expectations, the corresponding 
background composition and the signal efficiencies for the studied final states are shown 
in Table 8. In the bbl + l~z/z/ final state, one data event was retained after the final 
selection level, for a SM expectation of 1.5 ± 0.7 events. This event belonged to the no-id 
sample and was collected at yfs = 200 GeV. For all the other final states, discriminant 
analyses were used. In these discriminant variable, \a(Cs/ £>b), was defined. The 

distributions of Iyl(Cs/Cb), for the different analysis channels are shown in Fig. 9. No 
evidence for a signal was found in any of the channels and the full information, i. e. event 
numbers and the shapes of the distributions of the discriminant variables were used to 
derive limits on BR(h' -> bZ) and BR(h' -> cW). 
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final state 


data 

(hM ± stat. error) 


background 
composition (To) 
qq WW ZZ 77 


signal 

cv * / fry \ 

efficiency (%) 


bbl + E vv e sample 
(first selection \i sample 
level) no-id sample 


16 (13.2±0.8) 
14 (16.7±0.8) 
208 (191.0±3.0) 


16 16 68 
10 90 
8 80 12 


35.1±2.6 
53.4±2.7 
12.3±1.0 


bbqqz/z/ 


533 (511.7±8.3) 


76 17 2 5 


57.6±1.7 


bbqqqq 


1662 (1673.9±11.4) 


35 65 


66.0±1.5 


e sample 

ccqql + z/ fi sample 
no-id sample 
no lepton sample 


289 (278.8±3.5) 
293 (294.5±3.4) 
179 (176.8±2.8) 
533 (511.7±8.3) 


7 82 11 
2 97 1 
9 84 7 
76 17 2 5 


45.3±2.7 
56.4±2.7 
5.3±0.7 
8.9±0.9 


ccqqqq 


1662 (1673.9±11.4) 


35 65 


67.3±1.5 



Table 8: Summary of the total number of selected data events and SM expectations for 
the studied final states after the final selection (first selection level for bbl + Ez/z/). The 
corresponding background composition and signal efficiencies for my = 100 GeV/c 2 and 
= 205 GeV are also shown. 



5.1 Limits on BR(W -> bZ) and BR(W -> cW) 

Upper limits on the product of the e + e~ — > b'b' cross-section and the branching ratio 
as a function of the b' mass were derived at 95% confidence level (CL) in each of the 
considered b' decay modes (FCNC and CC), taking into account the values of the dis- 
criminant variables and their expected distributions for signal and background, the signal 
efficiencies and the data luminosities at the various centre-of-mass energies. 

Assuming the SM cross-section for the pair production of heavy quarks at LEP [7,14], 
these limits were converted into limits on the branching ratios corresponding to the 
b' — > bZ and b' — > cW decay modes. The modified frequentist likelihood ratio method [23] 
was used. The different final states and centre-of-mass energy bins were treated as inde- 
pendent channels. For each b' mass only the channels with ^/s > 2 my were considered. 
In order to avoid some non-physical fluctuations of the distributions of the discriminant 
variables due to the limited statistics of the generated events, a smoothing algorithm was 
used. The median expected limit, i.e. the limit obtained if the SM background was the 
only contribution in data, was also computed. In Fig. 10 the observed and expected limits 
on BR(h' — > bZ) and BR(b' — > cW) are shown as a function of the b' mass. The la and 
2(T bands around the expected limit are also shown. The observed and expected limits 
are statistically compatible. At 95% CL and for my = 96 GeV/c 2 , the BR(b' — > bZ) and 
BR(b' — > cW) have to be below 51% and 43%, respectively. These limits were evaluated 
taking into account the systematic uncertainties, as explained in the next subsection. 

The limits obtained for BR(h' — > bZ) are compatible with those presented by CDF [10] 
for a b' mass of 100 GeV/c 2 . Below this mass, the DELPHI result is more sensitive and 
the CDF limit degrades rapidly. For higher b' masses, the LEP-II kinematical limit is 
reached and the present analysis looses sensitivity. 



10 



5.2 Systematic uncertainties 

The evaluation of the limits was performed taking into account systematic uncertain- 
ties, which affect the background estimation, the signal efficiency and the shape of the 
distributions used. The following systematic uncertainties were considered: 

• SM cross-sections: uncertainties on the SM cross-sections translate into uncertainties 
on the expected number of background events. The overall uncertainty on the most 
relevant SM background processes for the present analyses is typically less than 
2% [24], which leads to relative changes on the branching ratio limits below 6%; 

• Signal generation: uncertainties on the final state quark hadronisation and fragmen- 
tation modelling were studied. The Lund symmetric fragmentation function was 
tested and compared with schemes where the b and c quark masses are taken into 
account [14]. This systematic error source was estimated to be of the order of 20% 
in the signal efficiency, by conservatively taking the maximum observed variation. 
The relative effect on the branching ratio limits is below 16%; 

• Smoothing: the uncertainty associated to the discriminant variables smoothing was 
estimated by applying different smoothing algorithms. The smoothing procedure 
does not change the number of SM expected events or the signal efficiency, but may 
lead to differences in the shape of the discriminant variables. The relative effect of 
this uncertainty on the limits evaluation was found to be below 9%. 

Further details on the evaluation of the systematic errors and the derivation of limits can 
be found in [25]. 

6 Constraints on Rckm 

The branching ratios for the b' decays can be computed within a four generations 
sequential model [5-7]. As discussed before, if the b' is lighter than both the t and the t' 
quarks and mi < my < mn, the main contributions to the b' width are BR(b' —> bZ) and 
BR(b' — > cW) [7]. Using the unitarity of the CKM matrix, its approximate diagonality 
(Kb' Kb ~ 0) and taking Kb ~ 1CT 2 [12], the branching fractions can be written as a 
function of three variables: Rckm = | v V fy | , m t / and my [5-7]. 

Fixing m t i — my, the limits on BR(b' — > bZ) and BR(b' — > cW) (Fig. 10) can be 
translated into 95% CL bounds on Rckm as a function of my. Two extreme cases were 
considered: the almost degenerate case, with m t i —my = 1 GeV/c 2 , and the case in which 
the mass difference is close to the largest possible value, m t i — my = 50 GeV/c 2 [3,5]. 
The results are shown in Fig. 11 and Fig. 12. In the figures, the upper curve was obtained 
from the limit on BR(b' — > cW), while the lower curve was obtained from the limit on 
BR(b' bZ), which decreases with growing m t r. This suppression is due to the GIM 
mechanism [26] as m t > approaches m t . On the other hand, as the b' mass approaches 
the bZ threshold, the b' — > bg decay dominates over b' — > bZ [7] and the lower limit on 
Rckm becomes less stringent. The expected limits on BR(b' — > bZ) did not allow to set 
exclusions for low values of Rckm and m t i — my = 1 GeV/c 2 (see Fig. 11). 

7 Conclusions 

The data collected with the DELPHI detector at y/s = 196—209 GeV show no evidence 
for the pair production of b'-quarks with masses ranging from 96 to 103 GeV/c 2 . 
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Assuming the SM cross-section for the pair production of heavy quarks at LEP, 95% 
CL upper limits on BR{b' — > bZ) and BR(b' — > cW) were obtained. It was shown that, at 
95% CL and for my = 96 GeV/c 2 , the BR{b' -> bZ) and BR{b' -> cW) have to be below 
51% and 43%, respectively. The 95% CL upper limits on the branching ratios, combined 
with the predictions of the sequential fourth generation model, were used to exclude 
regions of the (Rckm, my) plane for two hypotheses of the m t i — my mass difference. 
It was shown that, for m t , - my = 1 (50) GeV/c 2 and 96 GeV/c 2 < my < 102 GeV/c 2 , 
Rckm is bounded by an upper limit of 3.8 x 10~ 3 (1.2 x 10~ 3 ). For my = 100 GeV/c 2 and 
m t t — my = 50 GeV/c 2 , the CKM ratio was constrained to be in the range 4.6 x 10~ 4 < 
Rckm < 7.8 x 10" 4 . 
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b / s / d t' / t / c / u 

— K b — < 

Z/H/g/7 ^"w 
a) b) 

Figure 1: The Feynman diagrams corresponding to the b' (a) FCNC and (b) CC decay 
modes are shown. 




Figure 2: The final states associated to the b' (a) FCNC and (b) CC decay modes are 
shown. Only those states analysed here are indicated. 
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Figure 3: Data and SM expectation after the preselection level for the bbl + l~z/z/ final state 
and centre-of-mass energies above 200 GeV. (a) The angle between the most energetic 
lepton and the closest charged-particle track (e sample), (b) the missing momentum (/i 
sample) and (c) the momentum of the most energetic jet (no-id sample) are shown. The 
signal distributions for % = 100 GeV/c 2 and \fs = 205 GeV are also shown with 
arbitrary normalisation. The background composition is 11% of qq, 69% of WW, 15% of 
ZZ and 5% of 77 for the e sample, 6% of qq, 90% of WW and 4% of ZZ for the /1 sample 
and 45% of qq, 48% of WW, 5% of ZZ and 2% of 77 for the no-id sample. 
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Figure 4: Data and SM expectation after the first selection level for the bbl + l~z/i/ final 
state and for centre-of-mass energies above 200 GeV. (a) The momentum of the most 
energetic jet (e sample), (b) the angle between the two leptons (fi sample) and (c) the 
ratio between the missing momentum and missing energy (no-id sample) are shown. The 
signal distributions for my = 100 GeV/c 2 and yfs = 205 GeV are also shown with 
arbitrary normalisation. The arrows represent the cuts applied in the second selection 
level. 
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Figure 5: Comparison of data and SM expectation distributions of the \ 2 /n.d.j '. of the fit 
imposing energy-momentum conservation and no missing energy for the bbqqz^z/ final state 
at centre-of-mass energies above 200 GeV. The arrow shows the applied cut. The signal 
for my = 100 GeV/c 2 and ^/s = 205 GeV is also shown with arbitrary normalisation. 
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Figure 6: Variables used in the discriminant analysis (bbqqi/z/ final state). The data and 
SM expectation distributions for centre-of-mass energies above 200 GeV are shown for (a) 
the missing mass, (b) A^J^ x min(sin^j 1 , sin#j 2 ), where A^J^ is the acoplanarity and 0j 1 j 2 
are the polar angles of the jets when forcing the events into two jets, (c) the acollinearity 
between the two most energetic jets (with the event particles clustered into four jets), 
(d) the sum of the first and third Fox- Wolfram moments and (e) the polar angle of the 
missing momentum. The signal distributions for my = 100 GeV/c 2 and y/s = 205 GeV 
are also shown with arbitrary normalisation. 
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Figure 7: Variables used in the discriminant analysis (bbqqqq final state). The data 
and SM expectation for centre-of-mass energies above 200 GeV are shown for (a) 
— log 10 (2/4^3), (b) — log 10 (y 5 ^ 4 ), (c) the acollinearity between the two most energetic 
jets, with the events clustered into four jets (see text for explanation), (d) the hi + h3 
Fox- Wolfram moments sum, (e) the momentum of the most energetic jet and (f ) the angle 
between the two most energetic jets. The signal distributions for my = 100 GeV/c 2 and 
\/s = 205 GeV are also shown with arbitrary normalisation. 
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Figure 8: Variables used in the discriminant analysis (ccqql + z/ final state). The data 
events and background expectation for centre-of-mass energies above 200 GeV are shown 
for (a) the hi + h3 Fox- Wolfram moments sum (e sample), (b) the invariant mass of 
the two jets with the events clustered into two jets (e sample), (c) — log w (y^ 3 ) (p 
sample), (d) Yli \Pi\IV^i where pi are the momenta of the charged particles (excluding 
the lepton) in the same hemisphere as the lepton (p sample), (e) the acollinearity between 
the two most energetic jets (no-id sample) and (f) the angle between the lepton and the 
missing momentum (no-id sample). The signal distributions for my = 100 GeV/c 2 and 
\/s = 205 GeV are also shown with arbitrary normalisation. 
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Figure 9: Discriminant variables \a{Cs/ Cb) for data and SM simulation (centre-of- 
mass energies above 200 GeV). FCNC b' decay mode: (a) bbqqi/z/ and (b) bbqqqq. 
CC V decay mode: (c) ccqql + z/ (e sample), (d) ccqql + z/ (// sample), (e) ccqql + z/ {no-id 
sample) (f) ccqql + z/ (no lepton sample) and (g) ccqqqq. The signal distributions for 
my = 100 GeV/c 2 and y/s = 205 GeV are also shown with arbitrary normalisation. 



22 



DELPHI 



100 




103 



40 



20 



96 



100 



as 



20 - 



96 



a) b'— > bZ decay 



observed limit 
expected limit 
expected ± 1(5 
expected + 2(5 



100 



80 



60 



40 



20 



97 



98 



99 



100 



101 



102 103 

m,. (GeV/c ) 
102 103 




b) h'— » cW decay 



100 



- 80 



- 60 



40 

observed limit 
expected limit 

expected + la 20 
expected + 2(5 



97 



98 



99 



100 



101 



102 



103 







m, , (GeV/c) 



Figure 10: The observed and expected upper limits at 95% CL on (a) BR(b' — > bZ) and 
(b) BR(b' — > cW) are shown. The la and 2a bands around the expected limit are also 
presented. Systematic errors were taken into account in the limit evaluation. 
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Figure 11: The excluded region in the plane (Rckm, Tn\>') with m t / — my = 1 GeV/c 2 , 
obtained from the 95% CL upper limits on BR(h' -> bZ) (bottom) and 5i?(b' -> cW) 
(top) is shown. The light and dark shadings correspond to the observed and expected 
limits, respectively. The expected limits on BR(b' — > bZ) did not allow exclusions to be 
set for low values of Rckm- 
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Figure 12: The excluded region in the plane (Rckm, rny) with m t > — m h > = 50 GeV/c 2 , 
obtained from the 95% CL upper limits on BR(b' — > bZ) (bottom) and BR(b f — > cW) 
(top) is shown. The light and dark shadings correspond to the observed and expected 
limits, respectively. 



